Information Theoretic Interpretation of Deep learning
نویسندگان
چکیده
We interpret part of the experimental results of Shwartz-Ziv and Tishby [2017]. Inspired by these results, we established a conjecture of the dynamics of the machinary of deep neural network. This conjecture can be used to explain the counterpart result by Saxe et al. [2018].
منابع مشابه
Information Theoretic-Based Interpretation of a Deep Neural Network Approach in Diagnosing Psychogenic Non-Epileptic Seizures
The use of a deep neural network scheme is proposed to help clinicians solve a difficult diagnosis problem in neurology. The proposed multilayer architecture includes a feature engineering step (from time-frequency transformation), a double compressing stage trained by unsupervised learning, and a classification stage trained by supervised learning. After fine-tuning, the deep network is able t...
متن کاملWHY AND HOW TO APPLY QUANTUM LEARNING AS A NEW APPROACH TO IMPLEMENTATION THE CURRICULUM
The present study was philosophical and analytical research that examines quantum learning as an effective approach to the curriculum in a qualitative way. It explored books, published essays, and related studies, and took some advantages of online materials on the issue from domestic and foreign sources. Because of large body of data on the issue, only the relevant information was included. Da...
متن کاملAn Information-Theoretic Optimality Principle for Deep Reinforcement Learning
We methodologically address the problem of Qvalue overestimation in deep reinforcement learning to handle high-dimensional state spaces efficiently. By adapting concepts from information theory, we introduce an intrinsic penalty signal encouraging reduced Q-value estimates. The resultant algorithm encompasses a wide range of learning outcomes containing deep Q-networks as a special case. Differ...
متن کاملMyocardial fibrosis delineation in late gadolinium enhancement images of Hypertrophic Cardiomyopathy patients using deep learning methods
Introduction: Accurate delineation of myocardial fibrosis in Late Gadolinium Enhancement on Cardiac Magnetic Resonance (LGE-CMR) has a crucial role in the assessment and risk stratification of HCM patients. As this is time-consuming and requires expertise, automation can be essential in accelerating this process. This study aims to use Unet-based deep learning methods to automate the mentioned ...
متن کاملDetection of children's activities in smart home based on deep learning approach
Monitoring behavior of children in the home is the extremely important to avoid the possible injuries. Therefore, an automated monitoring system for monitoring behavior of children by researchers has been considered. The first step for designing and executing an automated monitoring system on children's behavior in closed spaces is possible with recognize their activity by the sensors in the e...
متن کامل